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SERVER AND METHOD FOR TRANSMITTING STREAMING MEDIA 
TO CLIENT THROUGH A CONGESTED NETWORK 

BACKGROUND OF THE INVENTION 

1 . Field of the invention 

The present invention is related to the field of data transmission over 
networks, and more specifically to devices and methods for adjusting a transmission 
rate of streaming media depending on available bandwidth. 

2. Description of the related art 

Networks are increasingly used to transmit data. The data can be for various 
media, such as audio or video, and is usually organized in groups that are called 
frames. When such data is transmitted continuously, it is also called streaming media. 
The transmission can either be for live, or for canned media. Either way, the 
transmission has to be in real-time, or in delayed real-time. 

A fundamental requirement for streaming media is that it has to be played at a 
constant rate. Packet networks, however, may transmit data asynchronously. 
Therefore, for streaming media to be transmitted over packet networks, strict timing 
requirements have to be met. Furthermore, where there are two related but separate 
streams of media (such as audio and video), they have to be time-synchronized in 
order to achieve high-quality media play out. 

A network channel should be used that has a higher bandwidth than what is 
required by the streaming media. The network, however, is occasionally subject to 
congestion, which constricts the bandwidth of the channel. This causes play out of 
streaming media to exhibit the problem of network jitter. This means that what is 
played out might be momentarily discontinued. 

Referring to FIG. 1, the problem of network jitter has been addressed in the art 
by introducing a de-jitter buffer at the receiving end. A server or proxy 110 receives 
data from a source, and stores it in a transmit buffer 1 14. The proxy 110 then 
transmits the stored data through a network 150 to a client or receiver 160. Typically, 
the client 160 includes a fixed size de-jitter receive buffer 162. 
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The transmit buffer 1 14 transmits data at the regular rate. It does not transmit 
any faster, because it if did, it would overflow the buffer 162. The buffer 162 first fills 
up to its size with data, and then starts playing out at the same rate as it is being filled. 
Momentary constrictions in bandwidth affect only the fullness of the buffer, not the 
play out, therefore diminishing network jitter. 

The buffer 162 does not work well where there are high-bit rate media 
streams, which are transmitted through heavily congested networks. In these cases 
network jitter still occurs, notwithstanding the buffer 162. 

Referring to FIG. 2, the phenomenon of network jitter is now described in 
more detail. Various quantities are plotted against a time axis, which is shown as a 
line 210. 

A solid, piece-wise linear graph 220 represents the time evolution of the 
available bandwidth of network 150. Notice that the graph 250 has a nominal value 
BI, but which is not always the case. Indeed, for a heavily congested network, the 
graph 250 has two main bandwidth constrictions, which are represented by "wells" A 
and B. These are instances where the available network bandwidth drops to values 
below BI. Well A is deeper than well B, but does not last as long. 

A dashed line 230 represents the instantaneous rate of receiving data through 
the network. In FIG. 2, the line 230 starts at a value fy, which is the steady state, 
assuming no prior interruptions in the transmission. The value fy is also the 
streaming media play out rate. That is how fast the data stream is being transmitted 
from the proxy 110. In addition, line 230 is always less than the instantaneous value 
of graph 220. For example, the line 230 tracks the graph 220, as it decreases in well 
A and well B. Immediately after the wells, however, the dashed line 230 increases, as 
formerly bottlenecked data is released from the network. After a long, xminterrupted 
regular receiving time, line 230 settles back at a value fy. 

A line 240 represents the instantaneous fullness of the buffer 162. In FIG. 2, 
initially the buffer 162 is at its nominal size fy, after a long time of network 
congestion. 

The fullness of receiving buffer 162 remains at approximately the same level, 
until the first dip occurs (well A). Line 240 follows with a corresponding well A', 
and then recovers. In more mathematical terms, line 240 is determined from a time 
integral of the difference between line 230 minus the value fy. For example, if well A 
did not reach below fy, line 240 would not exhibit well A' at all. 
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Similarly, due to well B, line 240 exhibits a well B\ and then recovers. Well 
B', however, reaches zero, with minor blips 246. This corresponds to a situation of a 
buffer under flow. Data is still arriving (line 230 is above zero), but slower than it is 
expected to be played (line 230 is less than value fy). Between time points 242, 244, 
play out is interrupted. This interruption is called network jitter. 

As will be appreciated, network jitter is a disruption in play out of the 
streaming media. If it is video, then the image stops, or flashes intermittently. If it is 
audio, then the sound is interrupted. In both cases, network jitter is annoying to the 
end user. 

BRIEF SUMMARY OF THE INVENTION 

The present invention overcomes the problems and limitations of the prior art. 

Generally, the present invention provides a system and a method that prevents 
network jitter, even in networks with seriously interrupted bandwidth. 

The invention senses the available bandwidth of the network. When the 
bandwidth is decreased, the invention transcodes the streaming media. This means 
that each frame is coded with fewer bits, and thus requires less time to be transmitted. 
This prevents buffer underflow at the receiving end. 

Optionally and preferably the transcoded data is also redimdancy encoded. 
So, if packets are dropped during the congestion, less information is not lost. 

The invention offers the advantage that network jitter is avoided. The user is 
not subjected to jitter, and does not miss out on the continuity of the streaming media. 

The invention will become more readily apparent from the following Detailed 
Description, which proceeds with reference to the drawings, in which: 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a server in the prior art transmitting data to a 
client over a network. 

FIG. 2 is a time diagram illustrating why in the arrangement of FIG. 1 there 
occurs the phenomenon of network jitter that disrupts play out. 

FIG. 3 is a block diagram of a server made according to the invention 
transmitting data over a network to a client. 

FIG. 4 is a flowchart illustrating a method according to the invention. 
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FIG. 5 is a time diagram illustrating why the phenomenon of network jitter is 
avoided in the arrangement of FIG. 3. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT(S) 

As has been mentioned, the present invention provides a server for 
transmitting streaming media to the network, that accounts for bandv^dth availability 
fluctuations. Various aspects and embodiments of the invention are now described in 
more detail. 

Referring to FIG, 3, a device according to the invention is a server 310, which 
is also known as a proxy 310. As shovra, the server 310 can be either a full server, or 
a portion of a server. The server 310 is for transmitting data over a network 150, such 
as a packet network, to a client 360. The client 360, which is also known as a receiver 
360, includes at least one de-jitter buffer 364. Optionally it also includes a variable bit 
rate decoder 366. 

The server 310 preferably includes a transmit buffer 314, that is also knovra as 
transmitting means 314. Buffer 3 14 is for buffering data received from a source, and 
for transmitting to the network 150 at the regular rate, such as at a fixed frame rate r 
required by the streaming media in question. The frame rate r may be, for example, 
100 frames per second for all audio, or 25 frames per second for video. The streaming 
media may have a constant or a variable transmission rate. When transmitting canned 
media, the transmit buffer 314 is not necessary. 

The server 310 moreover includes means for monitoring a bandwidth of the 
network 150. This feature can be optionally implemented in conjunction with control 
unit 335. This feature is used to provide an estimate of how fast data is being 
received at the client 360. This in tum is used to predict how full is the receive buffer 
364. 

The server 310 also includes a transcoder 340, made as is known in the art, 
which is otherwise knovra as transcoding means. The transcoder 340 is for 
transcoding the buffered streaming media, which is output through the transmit buffer 
314. This is preferably enabled only when the monitored bandv^dth of the network 
150 becomes less than a first preset bandwddth. Once transcoded, the same number of 
frames uses less bandwidth. 

The server 310 preferably also comprises a control unit 355. The control unit 
355 is primarily for controlling the transcoder 340. Controlling is mainly by 
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activating the transcoder 340 when the monitored bandwidth is less than the first 
preset value. In addition, the transcoder 340 is deactivated when the bandwidth of the 
network is restored to its full bandwidth. The control unit 355 can be implemented as 
a processor, software in a processor, a separate circuit, etc. 

The server 310 preferably also includes a redundancy encoder 350, which is 
also known as redundancy encoding means 350. The redundancy encoder 350 is for 
redundancy encoding the transcoded data, if the monitored bandwidth is less than a 
second preset value. Preferably the control unit 355 activates the redundancy encoder 
350 when the monitored bandwidth is less than the second preset value. The second 
preset value may be the same as the first preset value, although that is not necessary. 

It is readily apparent that the present invention can be implemented in logic 
circuitry, in a dedicated microcontroller circuit, or in a general purpose computer 
adapted with software. The invention also provides software that can practice 
methods of the invention. 

Referring to FIG. 4, a method of the invention is described with reference to 
flowchart 400. The method is for a server to retransmit streaming media to a network. 
The streaming media is encoded in data, and ultimately destined for a client. 

According to an optional box 410, a next portion of the streaming media is 
received in a server. According to a box 420, the received portion is stored in a 
transmit buffer of the server. 

According to a box 430, the available bandwidth of transmission is monitored. 
This can be accomplished by monitoring the available bandv^dth of the network. 

According to a box 440, it is inquired whether the monitored bandv^dth is less 
than a first preset bandwidth. That is a measure of how congested is the network. 

If it is, then according to box 450, the next portion of the streaming media is 
transcoded. It can be transcoded by being played out of the transmit buffer into a 
transcoder. Alternately, the data that encodes the streaming media can always be 
routed through the transcoder, which in turn is activated when a certain network 
congestion is determined. 

If the congestion is no longer detected, transcoding can stop. Nevertheless, it 
is suggested to continue transcoding for a while. This will give the opportimity for 
the bottlenecked transcoded frames to be entered into the receiving buffer. 

Continuing for box 450, according to a box 460, it is inquired whether the 
monitored bandwidth is less than a second preset bandwidth. The second preset 
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bandwidth can be the same as the first preset bandwidth, in which case the inquiry of 
box 460 is resolved at box 440. 

If the monitored bandwidth is less than the second preset bandwidth then, 
according to a box 470, the next portion is redundancy encoded. This is accomplished 
by having the nth packet contain the nth frame, as well as the transcoded (n-m)th 
frame. Even if packets are dropped during network congestion, and less than m 
consecutive packets are dropped, the receiver can still decode the frame using the 
redundant coded data. 

Redundancy encoding can be by routing the data through a redundancy 
encoder, and activating it when a certain network congestion is detected. Alternately, 
the data can be switched around a different path within the server. 

Execution then proceeds to box 480 where, whether transcoded or not, the 
next portion of the streaming media is transmitted to the network. 

The performance of the invention can be now appreciated with reference to 
FIG. 5. Various quantities are plotted against a time axis, which is shown as a line 
510. 

A solid, piece-wise linear graph 220 represents the time evolution of the 
available bandwidth of network 150. It vdll be observed that graph 220 is identical in 
time evolution to graph 220 of FIG. 2, with wells A and B. The similarity is 
intentional, so that the performance of the system of the invention can be compared to 
that of the prior art, xmder the same conditions. 

A dashed line 530 represents the instantaneous rate of receiving data through 
the network 150 in client 360. In FIG. 2, the line 530 starts at a value fy, which is the 
steady state, assuming no prior interruptions in the transmission. The value fy is 
again the streaming media play out rate, which is how fast the data stream is being 
transmitted from the server 310. 

Line 530 starts out less than the instantaneous value of graph 220. However, 
during the well A and well B, the monitoring means instructs the server 3 10 to 
activate the transcoder 340. The subsequently arriving data is transcoded. This 
means that the frames are each having less data, and are therefore being registered 
faster. So, when well A and well B start, line 530 dips below the line fy, but then 
compensates by going higher. After a long, iminterrupted regular receiving time, line 
530 settles back at a value fy. 
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A line 540 represents the instantaneous fullness of the buffer 364. In FIG. 5, 
initially the buffer 364 is at its nominal size fy, after a long time of network 
congestion. Again, line 540 is given by an integral of a time difference between line 
530 minus value fy. As such, it follows with two wells A", B" the wells A, B, 
respectively. In fact, when line 530 catches up vsdth value fy, that is when wells A", 
B" bottom out (have a local minimum). 

Notice that well B" never reaches zero. The buffer 364 does not xmderflow. 
This means that network jitter is avoided, where it was not avoided in the prior art of 
FIG. 2. 

During the congestion periods of wells A and B, however, the received data is 
transcoded, not regular. This is between the pair of time points 542 and 544 for well 
A", and the pair of time points 546, 548 for well B". 

When the transcoded data is played out, there is a reduction in quality. 
Nevertheless, the user does not feel he is missing out on any information, and may 
tolerate the play out during the congested time. 

A person skilled in the art will be able to practice the present invention in view 
of the description present in this document, which is to be taken as a whole. 
Numerous details have been set forth in order to provide a more thorough 
understanding of the invention. In other instances, well-known features have not been 
described in detail in order not to obscure urmecessarily the invention. 

While the invention has been disclosed in its preferred form, the specific 
embodiments thereof as disclosed and illustrated herein are not to be considered in a 
limiting sense. Indeed, it should be readily apparent to those skilled in the art in view 
of the present description that the invention can be modified in numerous ways. The 
inventor regards the subject matter of the invention to include all combinations and 
subcombinations of the various elements, features, functions and/or properties 
disclosed herein. 

The foUov^ng claims define certain combinations and subcombinations, which 
are regarded as patentable. Additional claims for other combinations and 
subcombinations of features, functions, elements and/or properties may be presented 
in this or a related document. 
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